Adjusting output image of image data

ABSTRACT

Using image data generated by an image generating device, and image generation record information that is associated with the image data and that includes at least subject area information indicating a subject area in the image, hue of each pixel in the subject area is calculated, and in the event that the proportion of pixels having hue of a predetermined color range is greater than a first predetermined threshold value, an image quality adjustment process appropriate for an image containing a subject identified by the predetermined color range is executed. In preferred practice, the predetermined color range is the skin color range, and the image quality adjustment process is a process appropriate for a portrait image. When the image quality adjustment process appropriate for a portrait image is not executed, and in the event that the proportion of pixels having hue in the green range is greater than a second predetermined threshold value or in the event that the proportion of pixels having hue in the sky blue range is greater than a third predetermined threshold value, an image quality adjustment process appropriate for a landscape image is executed.

TECHNICAL FIELD

The present invention relates to an image adjustment technique formaking adjustments to image quality of image data.

BACKGROUND ART

To date, the practice has been that the user selects a shooting scene,such as a portrait or landscape, with a camera or printer driver, andimage adjustment appropriate to the particular image is performedaccording to the scene selection (see, for example, JP11-136219A,JP2001-169135A, and JP2001-298631A).

However, with the prior art techniques, selection of a shooting scenesuch as a portrait or landscape requires the user to perform a fairlycomplicated operation during shooting or during printing.

For example, where shooting scene is selected with a camera, where aportrait image and a landscape image are shot in series, it becomesnecessary to reset the scene each time an image is taken. Where shootingscene is selected with a printer driver, it becomes necessary to selectthe shooting scene for each image, which involves a complicatedoperation in cases where portrait images and landscape images areintermixed.

In view of the problems mentioned above, it is an object of theinvention to automatically carry out image quality adjustmentsappropriate for individual image data.

DISCLOSURE OF THE INVENTION

To address the aforementioned issues at least in part, the output deviceaccording to the invention is an output device for outputting an imageusing image data generated by an image generating device, and imagegeneration record information that is associated with the image data andthat includes at least a subject area information indicating a subjectarea in the image, wherein the output device comprises an image qualityadjuster that calculates the hue of each pixel in the subject area, andin the event that the proportion of pixels having hue of a predeterminedcolor range is greater than a first predetermined threshold value,executes an image quality adjustment process appropriate for an imagecontaining a subject identified by the predetermined color range; and animage output unit that outputs an image according to thequality-adjusted image data.

With the output device according to the invention, since the decision asto whether to execute image quality adjustment appropriate for an imagecontaining a subject identified by the predetermined color range is madeautomatically on the basis of subject area information indicating asubject area in the image, and an image quality adjustment process iscarried out according to this decision, image quality adjustment to aparticular image data can be carried out automatically.

In preferred practice, in the aforementioned output device, thepredetermined color range may be the skin color range, and the imagequality adjustment process will be a process appropriate for a portraitimage.

With this arrangement, the decision as to whether to execute imagequality adjustment appropriate for a portrait image can be madeautomatically on the basis of subject area information indicating asubject area in the image, and an image quality adjustment processcarried out according to this decision.

It is possible to execute the image quality adjustment processappropriate for a portrait image, exclusively on a partial area thatincludes the subject area.

It is possible to execute image quality adjustment exclusively on atarget processing area which is a portion within the image, the targetprocessing area containing pixels present within the subject area andhaving color of the predetermined color range. The target processingarea may include pixels of a first type present within the subject areaand having color of the predetermined color range, and pixels of secondtype present outside the subject area, but contiguous with pixels of afirst type and having color of the predetermined color range.

In preferred practice, in the aforementioned output device, when animage quality adjustment process appropriate for a portrait image is notexecuted, the image quality adjuster may execute an image qualityadjustment process appropriate for a landscape image, in the event thatthe proportion of pixels having hue in the green range is greater than asecond predetermined threshold value, or the proportion of pixels havinghue in the sky blue range is greater than a third predeterminedthreshold value.

With this arrangement, in the event that image quality adjustmentappropriate for a portrait image is not executed, a decision as towhether to execute an image quality adjustment process appropriate for alandscape image can be made automatically on the basis of subject areainformation indicating a subject area in the image, and image qualityadjustment processing carried out according to this decision.

The invention may be realized in various embodiments, for example, animage output method and image output device; an image data processingmethod and image data processing device; a computer program forimplementing functions of such a method or device; a storage mediumhaving such a computer program stored thereon; and a data signalembodied in a carrier wave containing such a computer program.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing the arrangement of an image outputsystem as a embodiment of the invention.

FIG. 2 is a block diagram showing a simplified arrangement of a digitalstill camera 12.

FIG. 3 is an explanatory diagram conceptually illustrating an example ofan arrangement within an image file that can be used in this embodiment.

FIG. 4 is an explanatory diagram describing an example of data structureof an ancillary information storage field 103.

FIG. 5 is an explanatory diagram describing an example of data structurein an Exif data field.

FIG. 6 is an explanatory diagram showing a subject area 510 in an image500.

FIG. 7 is a block diagram showing a simplified arrangement of a printer20.

FIG. 8 is a block diagram showing an arrangement of printer 20, centeredon the control circuit 30 of printer 20.

FIG. 9 is a flowchart showing a process flow for generating an imagefile GF in a digital still camera 12.

FIG. 10 is a flowchart showing a processing routine for image processingin printer 20.

FIG. 11 is a flowchart showing a processing routine for image processingbased on image generation record information.

FIG. 12 is a flowchart showing a processing routine for Embodiment 1 ofan automatic image quality adjustment process.

FIGS. 13(A) and 13(B) illustrate a subject area in an image inEmbodiment 1, and a color histogram for pixels contained in the subjectarea.

FIG. 14 is an explanatory diagram showing the specifics of a processappropriate for a shooting scene in a embodiment of an automatic imagequality adjustment process.

FIG. 15 is a flowchart showing a processing routine for Embodiment 2 ofan automatic image quality adjustment process.

FIGS. 16(A) and 16(B) depict the processing in Embodiment 2.

FIG. 17 is an explanatory diagram showing an exemplary image outputsystem in which an image data processing may be implemented.

FIG. 18 is a flowchart showing an image processing routine with thecolor space conversion process omitted.

FIG. 19 is a flowchart showing another example of a processing routinefor image processing based on image generation record information.

FIG. 20 is a flowchart showing another example of a processing routinefor image processing based on image generation record information.

BEST MODE FOR CARRYING OUT THE INVENTION

Output image adjustment of image files according to the presentinvention is described hereinbelow through several embodiments, in thefollowing order, with reference to the accompanying drawings.

A. Arrangement of Image Output System

B. Arrangement of Image File

C. Arrangement of Image Output System Capable of Using Image File

D. Image Processing in Digital Still Camera

E. Image Processing in Printer

F. Embodiment of Automatic Image Quality Adjustment Process

G. Arrangement of Image Output System Employing Image Data ProcessingDevice

H. Variations

A. Arrangement of Image Output System

FIG. 1 illustrates an exemplary image output system in which an outputdevice may be implemented as a embodiment of the invention. Image outputsystem 10 comprises a digital still camera 12 as an image generatingdevice for generating image files; and a printer 20 as an image outputdevice. An image file generated by digital still camera 12 may betransferred to printer 20 via a cable CV, or by directly inserting intoprinter 20 a memory card MC on which the image file is stored. Printer20 executes image quality adjustment processing of image data based onthe image file that has been read in, and outputs an image. As theoutput device, a CRT display, LCD display, or other monitor 14, aprojector, or the like may be used in addition to or instead of aprinter 20. The following description is based on the use of a printer20 as the output device comprising an image quality adjuster and animage output unit, with a memory card MC being inserted directly intothe printer 20.

FIG. 2 is a block diagram showing a simplified arrangement of digitalstill camera 12. The digital still camera 12 of this embodimentcomprises an optical circuit 121 for collecting optical information; animage acquisition circuit 122 for controlling the optical circuit toacquire an image; an image processing circuit 123 for processing theacquired digital image; a flash 130 serving as a supplemental lightsource; and a control circuit 124 for controlling the various circuits.Control circuit 124 comprises memory, not shown. Optical circuit 121comprises a lens 125 for collecting optical information; an aperture 129for adjusting the amount of light; and a CCD 128 for converting opticaldata passing through the lens into image data.

Digital still camera 12 stores the acquired image on a memory card MC.The typical storage format for image data in digital still camera 12 isthe JPEG format, but other storage formats, such as TIFF format, GIFformat, BMP format, or RAW data format may be used instead.

Digital still camera 12 comprises a Select/Set button 126 for settingvarious shooting parameters; and a liquid crystal display 127. Liquidcrystal display 127 is used to preview a shot image, and when settingthe aperture or other parameters using the Select/Set button 126.

Once a shot has been taken by digital still camera 12, image data andimage generation record information are stored in the form of an imagefile on memory card MC. Image generation record information may includeaperture value and other parameter settings at the time of shooting(time of generation of image data) as described in detail hereinbelow.

B. Arrangement of Image File

FIG. 3 is an explanatory diagram conceptually illustrating an example ofan arrangement within an image file that may be used in this embodiment.Image file GF comprises an image data storage field 101 for storingimage data GD; and an image generation record information storage field102 for storing image generation record information GI. Image data GD isstored, for example, in JPEG format, while image generation recordinformation GI is stored, for example, in TIFF format (a format in whichdata and data fields are identified by tags). The terms file structureand data structure in this embodiment refer to the structure of the fileor data in the form in which the file or data etc. is stored in a memorydevice.

Image generation record information GI is information relating to animage when image data is generated or shot by digital still camera 12 orother such image generating device, and includes settings such as thefollowing.

-   -   Subject distance    -   Subject distance range    -   Subject area    -   Exposure time    -   Aperture value    -   ISO speed rating (ISO speed)    -   Shooting mode    -   Maker name    -   Model name    -   Gamma value

The image file GF of this embodiment may basically comprise theaforementioned image data storage field 101 and image generation recordinformation storage field 102; or may have a file structure according toan existing standardized file format. The following specific descriptionpertains to the case where the image file GF pertaining to thisembodiment conforms to the Exif file format.

An Exif file has a file structure in accordance with the digital stillcameran image file format specification (Exif); the specification hasbeen proposed by the Japan Electronics and Information TechnologyIndustries Association (JEITA). Similar to the conceptual diagram inFIG. 3, the Exif file format comprises a JPEG image data storage fieldfor storing image data in the JPEG format; and an ancillary informationstorage field for storing information of various kinds relating tostored JPEG image data. The JPEG image data storage field corresponds tothe image data storage field 101 in FIG. 3, and the ancillaryinformation storage field to the image generation record informationstorage field 102. The ancillary information storage field stores imagegeneration record information relating to a JPEG image, such as shootingdate/time, aperture value, and subject distance.

FIG. 4 is an explanatory diagram describing an example of data structureof ancillary information storage field 103. In the Exif file format,hierarchical tags are used to designate data fields. Each data fieldcontains within it a plurality of subordinate data fields identified bysubordinate tags. In FIG. 4, areas enclosed by rectangles representsingle data fields, with tag names noted at upper left. In thisembodiment, three data fields whose tag names are APP0, APP1, and APP6are contained. The APP1 data field contains within it two data fieldswith the tag names IFD0 and IFD1. The IFD0 data field contains threedata fields with the tag names PM, Exif, and GPS. Data and data fieldsare stored according to a prescribed address or offset value; address oroffset value can be searched by means of tag name. In the output device,data corresponding to desired information is acquired by means ofspecifying and address or offset value corresponding to the desiredinformation.

FIG. 5 is an explanatory diagram describing an example of data structure(data tag names and parameter values) in the Exif data field in FIG. 4,wherein tag names are referenced by tracing in the order APP1-IFD0-Exif.As shown in FIG. 4, the Exif data field may include a data field withthe tag name MakerNote; the MakerNote data field may in turn include aplurality of items of data, although these are omitted in FIG. 5.

As shown in FIG. 5, the Exif data field stores parameter values relatingto information such as subject area, subject distance, exposure time,aperture value, and ISO speed rate, etc. In this embodiment, subjectarea can be used as information indicating the area of a subject.

FIG. 6 shows a subject area 510 in an image 500. As shown in thedrawing, the subject area is represented by center coordinates with theupper left of the image as the origin, and the diameter of the area. Thesubject area may be rectangular in shape, in which case the area rangewill be represented in terms of height and width.

Information associated with image data may also be stored appropriatelyin fields other than the Exif data field in FIG. 4. For example, makername and model name are stored in the data field with the tag name IFD0,as information identifying the image generating device.

C. Arrangement of Image Output Device

FIG. 7 is a block diagram showing a simplified arrangement of printer 20in the embodiment. Printer 20 is a printer capable of image output, forexample, an ink jet printer that ejects ink of four colors, cyan C,magenta Mg, yellow Y, and black K, on a print medium to produce a dotpattern. An electrophotographic printer that transfers and fixes toneronto a print medium may also be used. In addition to the four colorsindicated above, light cyan LC which is lighter in density than cyan C,light magenta LM which is lighter in density than magenta Mg, and darkyellow DY which is darker in density than yellow Y, may be used as inks.The arrangement may employ black K only where monochromatic printing isperformed; or red R or green G may be used. The type of ink or tonerused may be selected depending on the characteristics of the image foroutput.

As shown in the drawing, printer 20 comprises a mechanism for driving aprint head 211 mounted on a carriage 21, to eject ink and form dots; amechanism for reciprocating carriage 21 in the axial direction of aplaten 23; a mechanism for feeding printer paper P by means of a paperfeed motor 24; and a control circuit 30. By means of these mechanisms,printer 20 functions as an image output unit. The mechanism forreciprocating carriage 21 in the axial direction of the platen 23 iscomposed of a slide rail 25 extending parallel to the axis of platen 23,for slidably retaining carriage 21; a pulley 27 linked via an endlessdrive belt 26 to a carriage motor 22; and a position sensor 28 forsensing the home position of carriage 21. The mechanism for feedingprinter paper P is composed of platen 23; paper feed motor 24 whichrotates platen 23; an auxiliary paper feed roller, not shown in thedrawing; and a gear train (not shown) for transmitting rotation of paperfeed motor 24 to platen 23 and the auxiliary paper feed roller.

Control circuit 30 exchanges signals with a printer control panel 29while appropriately controlling operation of paper feed motor 24,carriage motor 22, and print head 211. Printer paper P supplied toprinter 20 is set between platen 23 and the auxiliary paper feed roller,and advanced by a predetermined amount depending on the rotation angleof platen 23.

Carriage 21 has a print head 211, and enables mounting of an ink jetcartridge of utilizable inks. On the bottom face of print head 211 aredisposed nozzles for ejecting utilizable inks (not shown).

FIG. 8 is a block diagram showing an arrangement of printer 20, centeredon the control circuit 30 of printer 20. Within control circuit 30 aredisposed a CPU 31, PROM 32, RAM 33, a memory card slot 34 for acquiringdata from a memory card MC, a peripheral device input/output unit (PIO)35 for exchanging data with paper feed motor 24, carriage motor 22,etc., a drive buffer 37, and the like. Drive buffer 37 is used as abuffer for supplying dot on/off signals to print head 211. Thesecomponents are interconnected to a bus 38, enabling exchange of dataamong them. Control circuit 30 is also provided with a transmitter 39for outputting a drive waveform at predetermined frequency, and adistributed output device 40 for distributing the output of transmitter39 to print head 211 at predetermined timing.

Control circuit 30, while synchronizing with operations of paper feedmotor 24 and carriage motor 22, outputs dot data to drive buffer 37 atpredetermined timing. Control circuit 30 also reads an image file frommemory card MC, analyzes the ancillary information, and performs imageprocessing based on the image generation record information acquiredthereby. That is, control circuit 30 functions as the image qualityadjuster. The process flow of image processing executed by controlcircuit 30 will be described in detail hereinafter.

D. Image Processing in Digital Still Camera

FIG. 9 is a flowchart showing process flow for generating an image fileGF in digital still camera 12.

Control circuit 124 (FIG. 2) of digital still camera 12 generates imagedata GD in response to a shoot request, for example, depression of theshutter button (Step S100). Where aperture value, ISO speed, shootingmode and other parameter settings have been made, the set parametervalues are used when generating the image data GD.

Control circuit 124 stores the generated image data GD and imagegeneration record information GI as an image file GF on memory card MC(Step S110), and terminates the processing routine. Image generationrecord information GI includes parameters used at the time of imagegeneration, such as aperture value, ISO speed, etc.; shooting modeand/or other arbitrarily set parameter values; and parameter values setautomatically, such as maker name, model name, and the like. Image dataGD is stored in image file GF after being converted from the RGB colorspace to the YCbCr color space, and compressed in JPEG format.

By means of the aforementioned processes executed in digital stillcamera 12, both image data GD and image generation record information GIthat includes various parameter values at the time of generation ofimage data, are established in an image file GF for storage on memorycard MC.

E. Image Processing in Printer

FIG. 10 is a flowchart showing a processing routine for image processingin printer 20 of the embodiment. The following description is based on ascenario wherein a memory card MC having an image file GF stored thereonis inserted directly into printer 20. When memory card MC has beeninserted into memory card slot 34, the CPU 31 of control circuit 30(FIG. 8) of printer 20 reads out the image file GF from memory card MC(Step S200). Next, in Step S210, CPU 31 searches in the ancillaryinformation storage field of image file GF for image generation recordinformation GI indicating information at the time that the image datawas generated. In the event that image generation record information GIis found (Step S220: Y), CPU 31 acquires and analyzes the imagegeneration record information GI (Step S230). On the basis of theanalyzed image generation record information GI, CPU 31 executes imageprocessing, described hereinbelow (Step S240), outputs the processedimage (Step S250), and terminates the processing routine.

An image file created by a drawing application program or the like, onthe other hand, will not have image generation record information GIthat contains information such as aperture value and the like. If CPU 31does not find image generation record information GI (Step S200: N), itperforms standard processing (Step S260), outputs the processed image(Step S250), and terminates the processing routine.

FIG. 11 is a flowchart showing a processing routine for image processing(corresponding to Step S240 in FIG. 10) based on image generation recordinformation. The CPU 31 of control circuit 30 (FIG. 8) of printer 20reads out image data GD from the read out image file GF (Step S300)

As mentioned previously, digital still camera 12 stores image data GD asJPEG format files, and in a JPEG format file image data is stored usingan YCbCr color space. In Step S310, CPU 31 executes an operation using3×3 matrix S to convert image data based on an YCbCr color space intoimage data based on an RGB color space. This matrix operation isrepresented by the following arithmetic expression, for example.$\begin{matrix}{{\begin{pmatrix}R \\G \\B\end{pmatrix} = {S\begin{pmatrix}Y \\{{Cb} - 128} \\{{Cr} - 128}\end{pmatrix}}}{S = \begin{pmatrix}1 & 0 & 1.40200 \\1 & {- 0.34414} & {- 0.71414} \\1 & 1.77200 & 0\end{pmatrix}}} & \lbrack {{Eq}.\quad 1} \rbrack\end{matrix}$

Where the color space of image data generated by digital still camera 12is wider than a predetermined color space, for example, the sRGB colorspace, image data based on the RGB color space obtained in Step S310 mayin some instances contain useful data that is outside the defined areaof the RGB color space. Where image generation record information GIinstructs this out-of-defined area data to be handled as valid data, theout-of-defined area data will be kept as-is, while continuing withsubsequent image processing. In the absence of an instruction to handleout-of-defined area data as valid data, out-of-defined area data isclipped to the defined area. For example, where the defined area is0-255, negative value data of less than 0 is rounded to 0, and dataabove 256 to 255. In the event that the color space that is reproducibleby the image output unit is not wider than a predetermined color space,for example, the sRGB color space, it is preferable to perform clippingto the defined area regardless of any instruction in the imagegeneration record information GI. Such instances may include, forexample, cases where the image is output to a CRT, whose reproduciblecolor space is the sRGB color space.

Next, in Step S320, CPU 31 performs gamma correction and an operationemploying a matrix M, to convert image data based on an RGB color spaceto image data based on an XYZ color space. Image file GF may containgamma value and color space information at the time of image generation.In the event that image generation record information GI includes thisinformation, CPU 31 acquires the gamma value of the image data from theimage generation record information GI, and executes a gamma conversionprocess of the image data using the acquired gamma value. CPU 31 thenacquires color space information for the image data from the imagegeneration record information GI, and executes a matrix operation of theimage data using a matrix M that corresponds to the color space. In theevent that image generation record information GI does not contain agamma value, a gamma conversion process can be executed using a standardgamma value. In the event that image generation record information GIdoes not contain color space information, matrix operations may beexecuted using a standard matrix M. A gamma value and matrix for thesRGB color space may be used respectively as this standard gamma valueand matrix M. The matrix operation may be given by the followingarithmetic expression, for example. $\begin{matrix}{{\begin{pmatrix}X \\Y \\Z\end{pmatrix} = {M\begin{pmatrix}{Rt}^{\prime} \\{Gt}^{\prime} \\{Bt}^{\prime}\end{pmatrix}}}{M = \begin{pmatrix}0.6067 & 0.1736 & 0.2001 \\0.2988 & 0.5868 & 0.1144 \\0 & 0.0661 & 1.1150\end{pmatrix}}{{Rt},{Gt},{{Bt} \geq 0}}\begin{matrix}{{Rt}^{\prime} = ( \frac{Rt}{255} )^{\gamma}} & {{Gt}^{\prime} = ( \frac{Gt}{255} )^{\gamma}} & {{Bt}^{\prime} = ( \frac{Bt}{255} )^{\gamma}}\end{matrix}{{Rt},{Gt},{{Bt} < 0}}\begin{matrix}{{Rt}^{\prime} = {- ( \frac{- {Rt}}{255} )^{\gamma}}} & {{Gt}^{\prime} = {- ( \frac{- {Gt}}{255} )^{\gamma}}} & {{Bt}^{\prime} = {- ( \frac{- {Bt}}{255} )^{\gamma}}}\end{matrix}} & \lbrack {{Eq}.\quad 2} \rbrack\end{matrix}$

The color space of image data obtained after the matrix operation hasbeen executed is an XYZ color space. The XYZ color space is an absolutecolor space, and is a device-independent color space not dependent on adevice such as a digital still camera or printer. Thus,device-independent color matching may be carried out by means of colorspace conversion to the XYZ color space.

Next, in Step S330, CPU 31 performs an operation employing a matrix N⁻¹,and inverse gamma correction to convert image data based on the XYZcolor space to image data based on the wRGB color space. During inversegamma correction, CPU 31 acquires a printer-side gamma value from PROM32, and executes inverse gamma correction of the image data using theinverse of the acquired gamma value. CPU 31 then acquires from PROM 32 amatrix N⁻¹ that corresponds to the conversion from the XYZ color spaceto the wRGB color space, and performs a matrix operation on the imagedata using this matrix N⁻¹. This matrix operation may be given by thefollowing arithmetic expression, for example. $\begin{matrix}{{\begin{pmatrix}{Rw} \\{Gw} \\{Bw}\end{pmatrix} = {N^{- 1}\begin{pmatrix}X \\Y \\Z\end{pmatrix}}}{N^{- 1} = \begin{pmatrix}3.30572 & {- 1.77561} & 0.73649 \\{- 1.04911} & 2.1694 & {- 1.4797} \\0.06568289 & {- 0.241078} & 1.24898\end{pmatrix}}\begin{matrix}{{Rw}^{\prime} = ( \frac{Rw}{255} )^{1/\gamma}} & {{Gw}^{\prime} = ( \frac{Gw}{255} )^{1/\gamma}} & {{Bw}^{\prime} = ( \frac{Bw}{255} )^{1/\gamma}}\end{matrix}} & \lbrack {{Eq}.\quad 3} \rbrack\end{matrix}$

Next, in Step S340, CPU 31 executes automatic adjustment processing ofimage quality. Automatic image quality adjustment processing in theembodiment involves automatic image quality adjustment processing ofimage data using image generation record information (subject areainformation) contained in the image file GF. Automatic image qualityadjustment processing is described hereinbelow.

Next, in Step S350, CPU 31 executes a CMYK color conversion process anda halftone process for the purpose of printing. In the CMYK colorconversion process, CPU 31 refers to a look-up table (LUT), stored inPROM 32, for conversion from the wRGB color space to the CMYK colorspace, and converts the color space of the image data from the wRGBcolor space to the CMYK color space. That is, image data consisting ofRGB tone level values is converted to image data for use by printer 20,consisting, for example, of tone level values for six colors, C (Cyan),M (Magenta), Y (Yellow), K (Black), LC (Light Cyan), and LM (LightMagenta).

In the halftone process, CPU 31 executes a so-called halftone process toproduce halftone image data from the color-converted image data. Thishalftone image data is sequenced in the order of transmission to drivebuffer 37 (FIG. 8) to produce final print data, whereupon the processingroutine terminates. Image data processed by means of this processingroutine is output in Step S250 of the image processing routine shown inFIG. 10.

F. Embodiment of Automatic Image Quality Adjustment Process

FIG. 12 is a flowchart showing a processing routine for Embodiment 1 ofan automatic image quality adjustment process, corresponding to StepS340 in FIG. 11. CPU 31 (FIG. 8) analyzes the image generation recordinformation GI and acquires subject area information (Step S400). Next,in Step S410, CPU 31 calculates the hue H of each pixel in the subjectarea. In this embodiment, in order to calculate hue H, a conversion froman RGB color system to an HSI (Hue, Saturation, Intensity) color systemis performed. The HSI color system is preferred since hue H andintensity I are independent from each other, so that hue is unaffectedby image brightness. Some other appropriate color system, such as an HSV(Hue, Saturation, Value) color system or HSL (Hue, Saturation,Luminance) color system, may be used instead.

Equations for calculating hue H in the HIS color system are nowdiscussed. Where I=max {R, G, B} and i=min {R, G, B}, hue H will beundefined, or achromatic, when I=0; and when I≠0, they can be calculatedas shown by the following equations. $\begin{matrix}{\begin{matrix}{{{{when}\quad R} = I},} & {H = {\frac{\pi}{3}( \frac{G - B}{I - i} )}}\end{matrix}\begin{matrix}{{{{when}\quad G} = I},} & {H = {\frac{\pi}{3}( {2 + \frac{B - R}{I - i}} )}}\end{matrix}\begin{matrix}{{{{when}\quad B} = I},} & {H = {\frac{\pi}{3}( {4 + \frac{R - G}{I - i}} )}}\end{matrix}} & \lbrack {{Eq}.\quad 4} \rbrack\end{matrix}$

However, when hue H<0, 2π is added to the calculated value for H. Thevalue range for hue H is 0 −2π; in this embodiment, hue H is representedby a value range of 0°-360°.

Next, for all pixels within the subject area, frequencies of pixelshaving hues contained respectively in the skin color, green, and skyblue color ranges are calculated (Step S420). FIGS. 13(A) and 13(B) showa subject area in an image, and a color histogram for pixels containedin the subject area. Specifically, a portrait image is shown in FIG.13(A), and a histogram for a portrait image is shown in FIG. 13(B). Inthis embodiment, the hue range from 0° to 30° is designated as the skincolor range SR; the range from 100° to 130° is designated as the greencolor range GR, and the range from 230° to 260° is designated the skyblue color range BR. Hue ranges need not necessarily be the rangesdescribed above, and different ranges may be established instead.

Next, in Step S430, the proportion r(Skin) of skin color pixels relativeto the total number of pixels in the subject area is calculated, and adecision is made as to whether this proportion r(Skin) is greater than afirst threshold value Th1. In the event of a decision that r(Skin)>Th1,i.e., that the image is judged to be a portrait, the routine proceeds toStep S440 wherein processing appropriate to a portrait scene isexecuted, and the automatic image quality adjustment process thenterminated.

If on the other hand it is not true that r(Skin)>Th1, the routineproceeds to Step S450 wherein a decision is made as to whether theproportion r(Green) of green pixels relative to the total number ofpixels in the subject area is greater than a second threshold value Th2.In the event of a decision that r(Green)>Th2, i.e., that the image isjudged to be a landscape containing trees or woodlands, the routineproceeds to Step S470 wherein processing appropriate to a landscapescene is executed, and the automatic image quality adjustment processthen terminated.

If it is not true that r(Green)>Th2, the routine proceeds to Step S460wherein a decision is made as to whether the proportion r(Sky) of skyblue pixels relative to the total number of pixels in the subject areais greater than a third threshold value Th3. In the event of a decisionthat r(Sky)>Th3, i.e., that the image is judged to be a landscapecontaining sky, the routine proceeds to Step S470 wherein processingappropriate to a landscape scene is executed, and the automatic imagequality adjustment process then terminated; if it is not true thatr(Sky)>Th3, the routine proceeds to Step S480 wherein processingappropriate to a standard scene is executed, and the automatic imagequality adjustment process then terminated.

The aforementioned threshold values Th1, Th2, Th3 are preferablyestablished so as to give a high degree of accuracy in the decisions ofSteps S430, S450, and S460. It is also possible to establishpredetermined values as initial values, which can then be modified bythe user.

In this embodiment, processing appropriate to each scene is executed inthe manner shown in FIG. 14. For example, in the event of a decisionthat a scene is a portrait image, contrast is set to a somewhat weakerlevel, lightness to a somewhat lighter level, color balance to astandard level, saturation to a somewhat lower level, and sharpness to asomewhat weaker level. Also, since skin color is indicated as a memorycolor, skin color correction is executed using pre-stored skin colordata. Noise reduction is off. By executing image quality adjustmentprocessing in this way, a soft ambience may be imparted to the image,and human skin tones can be adjusted to desirable skin color. However,as regards image quality adjustment, it is not necessary to follow thesettings given in FIG. 14, it being possible to use different settingsinstead. In Embodiment 1 described hereinabove, the entire image isprocessed by the automatic image quality adjustment, but it is possibleinstead to process only the subject area. Alternatively, it is possibleto process a partial area that includes the subject area.

In this Embodiment 1, a shooting scene is selected automatically forindividual image data during the automatic image quality adjustmentprocess, and the appropriate image quality processing is carried out forthe image data. A resultant advantage is that there is no need for acomplicated manual operation by the user when shooting or when printing.

FIG. 15 is a flowchart showing a processing routine for Embodiment 2 ofan automatic image quality adjustment process. FIGS. 16(A) and 16(B)depict the processing in Embodiment 2. In Step S400, as in Embodiment 1,information relating to a subject area 510 a (FIG. 16(A)) is acquiredfrom the image generation record information. In the example of FIG.16(A), subject area 510 a is rectangular. In Step S401, the color rangeof the subject area is determined through analysis of the color of thesubject area. Here, “color range of the subject area” refers to a colorrange of pixels that make up the majority in the subject area (i.e., therepresentative color range of the subject area). As the method forcalculating the subject area color range, there may be employed, forexample, a method analogous to that of Embodiment 1, in which the colorof each pixel is converted to the HSI color system, a histogram relatingto hue H is calculated (FIG. 13(B)), and the range with the highestfrequency is designated as the color range of the subject area.Alternatively, average color for the subject area may be calculated, andthe color range of the subject area may be defined to be a rangecentered on this average color and having a predetermined width. In theexample of FIG. 16(A), since the subject area 510 a is established inthe face of the human figure, a skin color range SR like that shown inFIG. 13(B) will be determined as the color range of the subject area.Also, the subject area color range may be one selected from among aplurality of color range candidates (e.g., skin color range SR, greencolor range GR, or sky blue color range SR), or a representative colorrange calculated from pixel colors in the actual subject area.

Hereinafter, color within the color range of a subject area will betermed “subject color” and a pixel having subject color will be termed a“subject color pixel”. In Step S402, a processing target area isdetermined by searching for subject color pixels present within thesubject area and other subject color pixels contiguous therewith butoutside the subject area, and by defining the target area consisting ofthese subject color pixels. FIG. 16(B) shows a processing target area512, indicated by hatching, obtained in this manner. In this example,since the color range of subject area 510 a is skin color, subject area510 a is searched for pixels having skin color. Of subject color pixelspresent outside the subject area, those pixels that are contiguous withsubject color pixels inside the subject area 510 a are also extracted aspixels making up the processing target area 512. Here, “contiguous”pixels refers to pixels that are in contact in either the vertical orhorizontal directions. Here, it should be noted that if the subject area510 a is set larger than the face, processing target area 512 will besmaller than subject area 510 a.

The processing target area 512 may be composed of subject area colorpixels exclusively; alternatively, all pixels included within theoutermost contours of an area which is composed of subject area colorpixels may be designated as the processing target area 512. With theformer method, black eye areas present in the face, for example, willnot be included in the processing target area 512, whereas with thelatter method, these black eye areas may be included in the processingtarget area 512. An advantage of determining the processing target area512 by means of the former method is that excessive modification of thecolor of pixels having color other than subject area color can beprevented. On the other hand, an advantage of determining the processingtarget area 512 by means of the latter method is that image qualitywithin the processing target area 512 can be kept well-balanced evenafter image quality adjustment.

In Step S403, the processing target area 512 determined in the mannerdescribed above is subjected to image quality adjustment appropriate tothe color range of the subject area. Specifically, where the subjectcolor range is the skin color range, a process appropriate for aportrait scene (the process of Step S440 of FIG. 12) is executed. Wherethe subject color range is green or sky blue, a process appropriate fora landscape scene (the process of Step S470 of FIG. 12) is executed.Where the subject color range is some other color range, a processappropriate for a standard scene (the process of Step S480 of FIG. 12)is executed.

In this way, in Embodiment 2, the color range of a subject area isdetermined, and an image quality adjustment process is then executed ona processing target which is an area that includes pixels within thesubject area having color within this subject color range and otherpixels having subject color and contiguous therewith, whereby it ispossible to perform image quality adjustment appropriate for the colorrange of the subject area, on only a portion of the image. In Embodiment2 there is an additional advantage in that when, for example, a secondskin tone area is contained at a location away from the subject area,image quality adjustment in proximity to the subject area can be carriedout according to the color range of the subject area, without affectingthe color of this second skin tone area.

G. Arrangement of Image Output System Employing Image Data ProcessingDevice

FIG. 17 is an explanatory diagram showing an example of an image outputsystem in which an image data processing device may be implemented byway of a embodiment of the invention. Image output system 10B comprisesa digital still camera 12 as an image generating device for generatingimage files; a computer PC for executing image quality adjustmentprocessing based on an image file; and a printer 20B as an image outputdevice for outputting images. Computer PC is computer of commonly usedtype, and functions as an image data processing device. As the imageoutput device, a CRT display, LCD display, or other monitor 14B, aprojector, or the like may be used instead of printer 20B. In thefollowing description, it is assumed that a printer 20B is used as theimage output device. This embodiment differs from the image outputsystem described previously (FIG. 1) in that the image data processingdevice comprising the image quality adjuster is independent of the imageoutput device comprising the image output unit. The computer PC servingas the image data processing device, and the printer comprising theimage output unit as a whole can be termed an “output device” in thebroad sense.

An image file created in digital still camera 12 is transferred tocomputer PC via a cable CV, or by directly inserting into computer PC amemory card MC having the image file stored thereon. Computer PCexecutes image quality adjustment processing of the image data, based onthe image file that has been read in. The image data produced by theimage quality adjustment process is transferred to printer 20B via acable CV, and is then output by printer 20B.

Computer PC comprises a CPU 150 that executes a program for realizingthe aforementioned image quality adjustment process; RAM 151 fortemporarily storing results of operations by CPU 150, image data, andthe like; and a hard disk drive (HDD) 152 for storing data needed forimage quality adjustment processing, such as an image quality adjustmentprocessing program, lookup table, aperture value table, and the like.CPU 150, RAM 151, and HDD 152 function as the image quality adjuster.Computer PC further comprises a memory card slot 153 for installing amemory card MC; and an input/output terminal 154 for connecting aconnector cable from digital still camera 12 or the like.

An image file GF generated by a digital still camera 12 is supplied tocomputer PC via a cable or via a memory card MC. When an image dataprocessing application program, either an image retouching applicationor a printer driver, is run under user control, CPU 150 executes animage processing routine (FIG. 10) to process the image file GF whichhas been read in. Alternatively, the image data processing applicationprogram may be set to start up automatically when a memory card MC isinserted into memory card slot 153, or when connection of a digitalstill camera 12 to input/output terminal 154 via a cable is detected.

Image data processed by CPU 150, rather than being output in Step S250of the image processing routine (FIG. 10), is instead transferred to animage output device, for example, printer 20B, whereupon the imageoutput device receiving the image data executes image output.

In this embodiment, since image processing is carried out using an imagequality adjuster provided to computer PC, it is possible to use an imageoutput device that does not have an image quality adjuster. Where theimage output device per se comprises an image quality adjuster, imagedata may be output to the image output device without being subjected toimage processing on computer PC, with image processing instead beingcarried out by the image quality adjuster of the image output device.

As described hereinabove, in the preceding embodiments, the shootingscene can be identified automatically, and the appropriate image qualityadjustment can be executed depending on this identification, whereby itis a simple matter to obtain output results of high quality.

H. Variations

The invention is not limited to the embodiments set forth hereinabove,but may be reduced to practice in various embodiments without departingfrom the scope and spirit thereof. The following variations arepossible, for example.

H1. Variation 1

In the embodiments hereinabove, it is preferable that the automaticimage quality processing is not performed if an image file GF includesshooting mode information and if the shooting mode identified prior toproceeding to the automatic image quality processing routine is a modeother than the standard scene mode. By so doing, duplication of similarimage quality processing can be avoided in cases where the user has setthe shooting mode manually at the time of shooting.

H2. Variation 2

In the embodiments hereinabove, the automatic image quality processingis executed on all pixels included within a subject area; however, andarrangement wherein the user can select whether to execute automaticimage quality processing would also be possible.

H3. Variation 3

It is preferable that white balance is adjusted prior to executing theautomatic image quality process described in FIGS. 12-16. By so doing,deviation in hue due to differences in color temperature of the lightsource can be corrected. This will improve the accuracy of determinationin Steps S430, S450 and S460 in FIG. 12.

H4. Variation 4

Where an image file GF does not include image data gamma value or colorspace information, the color space conversion process of the imageprocessing routine shown in FIG. 11 (Step S320 and Step S330) may beomitted. FIG. 18 is a flow chart showing an image processing routinefrom which the color space conversion process has been omitted. Imagedata acquired in Step S500 is converted from image data based on a YCbCrcolor space to data based on an RGB color space, in Step S510. Next, inStep S520, automatic image quality adjustment processing is executedusing the image data obtained in Step S510. Next, in Step S530, a CYMKconversion process and halftone process for printing are executed.

H5. Variation 5

In the preceding embodiments, automatic image quality adjustmentprocessing is executed after color space conversion has been executed;but instead, color space conversion (Steps S320, S330) could be executedafter executing automatic image quality adjustment processing (StepS340).

H6. Variation 6

In the preceding embodiments, a printer is used as the image outputunit; however, an image output unit other than a printer could be usedinstead. FIG. 18 is a flowchart showing a processing routine for imageprocessing based on image generation record information, where a CRT isused as the image output unit. In contrast to the flowchart in FIG. 11,which depicts the use of a printer as the image output unit, the CYMKconversion process and halftone process for printing are omitted. Also,since a CRT can reproduce an RGB color space of image data obtained byexecuting a matrix operation (S), the color space conversion process isomitted as well. Where image data based on an RGB color space obtainedin Step S610 includes data outside the defined area of the RGB colorspace, the out-of-defined-area data is clipped, and then step S620 isexecuted. Where the color space utilizable by an image output unit isdifferent from an RGB color space, a color conversion process to a colorspace utilizable by the image output unit is executed in a manneranalogous to executing the CMYK color conversion process when using aprinter, and the resultant image is output by the image output unit.

H7. Variation 7

In the preceding embodiments, the description related to an Exif formatfile as a specific example of the image file GF; however, the format ofthe image file pertaining to the present invention is not limited tothis, and may be of any other format. Typically, it is sufficient forthe image file to include image data generated by an image generatingdevice, and image generation record information GI describing conditionsor information at the time of generation of the image data. With such afile, the image quality of image data generated by an image generatingdevice can be appropriately adjusted automatically, for output from anoutput device. The subject area is not limited to the Exif formatparameter described previously, and can be represented by data orparameters of various forms. For example, a parameter indicating a focuslocation or a focus area as a result of auto-focusing may be used. Wherethe user is able to indicate subject area location and shape duringshooting, parameters indicating these may be used as parametersindicating a subject area.

H8. Variation 8

Values of matrices S, N⁻¹, and M in the equations are merely exemplary,and can be modified appropriately depending on color space on which theimage file is based, the color space utilizable by the image outputunit, or the like.

H9. Variation 9

In the preceding embodiments, the description relates to the use of adigital still camera 12 as the image generating device, but image filesmay be generated using a different image generating device, such as ascanner, digital video camera, or the like.

H10. Variation 10

In the preceding embodiments, the description takes the example of acase where image data GD and image generation record information GI arecontained in the same image file GF, but image data GD and imagegeneration record information GI need not necessarily be stored withinthe same file. That is, it is sufficient for image data GD and imagegeneration record information GI to be associated with each other; forexample, it is acceptable to generate associating data that associatesimage data GD with image generation record information GI; store one orseveral sets of image data and image generation record information GI inindependent files; and refer to the associated image generation recordinformation GI when processing the image data GD. This is because, inthis case, although the image data GD and image generation recordinformation GI are stored in separate files, at the point in time ofimage processing utilizing the image generation record information GI,the image data GD and image generation record information GI are ininseparably linked, and thus function in substantially the same way asif stored in the same file. That is, the term image file GF in theembodiment includes also a format wherein image data GD and imagegeneration record information GI are associated, at least at the pointin time that image processing takes place. Also included are motionvideo files stored on optical disk media such as CD-ROM, CD-R, DVD-ROM,DVD-RAM, and the like.

INDUSTRIAL APPLICABILITY

The invention is applicable to printers, digital cameras, and computershaving image processing functionality.

1. An output device for outputting an image using image data generatedby an image generating device, and image generation record informationthat is associated with the image data and that includes at leastsubject area information indicating a subject area in the image, theoutput device comprising: an image quality adjuster that calculates ahue of each pixel in the subject area, and if a proportion of pixelshaving hue of a predetermined color range is greater than a firstpredetermined threshold value, executes an image quality adjustmentprocess appropriate for an image containing a subject identified by thepredetermined color range; and an image output unit that outputs animage according to the quality-adjusted image data.
 2. An output deviceaccording to claim 1, wherein the predetermined color range is a skincolor range, and the image quality adjustment process is a processappropriate for a portrait image.
 3. An output device according to claim2, wherein the image quality adjustment process appropriate for aportrait image is executed exclusively on a partial area that includesthe subject area.
 4. An output device according to claim 2 or 3, whereinwhen the image quality adjustment process appropriate for a portraitimage is not executed, and if a proportion of pixels having hue in agreen range is greater than a second predetermined threshold value or ifa proportion of pixels having hue in a sky blue range is greater than athird predetermined threshold value, the image quality adjuster executesan image quality adjustment process appropriate for a landscape image.5. An output device according to claim 1, wherein the image qualityadjustment is executed exclusively on a target processing area which isa portion of the image and which contains pixels present within thesubject area and having color of the predetermined color range.
 6. Anoutput device according to claim 5, wherein the target processing areaincludes pixels of a first type present within the subject area andhaving color of the predetermined color range, and pixels of a secondtype present outside the subject area, but contiguous with the pixels ofthe first type and having color of the predetermined color range.
 7. Animage data processing device for processing image data, using image datagenerated by an image generating device, and image generation recordinformation that is associated with the image data and that includes atleast subject area information indicating a subject area in the image,the image data processing device comprising: an image quality adjusterthat calculates a hue of each pixel in the subject area, and if aproportion of pixels having hue of a predetermined color range isgreater than a first predetermined threshold value, executes an imagequality adjustment process appropriate for an image containing a subjectidentified by the predetermined color range.
 8. An image data processingdevice according to claim 7, wherein the predetermined color range is askin color range, and the image quality adjustment process is a processappropriate for a portrait image.
 9. An image data processing deviceaccording to claim 8, wherein the image quality adjustment processappropriate for a portrait image is executed exclusively on a partialarea that includes the subject area.
 10. An image data processing deviceaccording to claim 8 or 9, wherein when the image quality adjustmentprocess appropriate for a portrait image is not executed, and if aproportion of pixels having hue in a green range is greater than asecond predetermined threshold value or if a proportion of pixels havinghue in a sky blue range is greater than a third predetermined thresholdvalue, the image quality adjuster executes an image quality adjustmentprocess appropriate for a landscape image.
 11. An image data processingdevice according to claim 7, wherein the image quality adjustment isexecuted exclusively on a target processing area which is a portion ofthe image and which contains pixels present within the subject area andhaving color of the predetermined color range.
 12. An image dataprocessing device according to claim 11, wherein the target processingarea includes pixels of a first type present within the subject area andhaving color of the predetermined color range, and pixels of a secondtype present outside the subject area, but contiguous with the pixels ofthe first type and having color of the predetermined color range.
 13. Animage quality adjustment method for adjusting image quality of imagedata, using image data generated by an image generating device, andimage generation record information that is associated with the imagedata and that includes at least subject area information indicating asubject area in the image, the method comprising the steps of:calculating a hue of each pixel in the subject area; and if a proportionof pixels having hue of a predetermined color range is greater than afirst predetermined threshold value, executing an image qualityadjustment process appropriate for an image containing a subjectidentified by the predetermined color range.
 14. An image qualityadjustment method according to claim 13, wherein the predetermined colorrange is a skin color range, and the image quality adjustment process isa process appropriate for a portrait image.
 15. An image qualityadjustment method according to claim 14, wherein the image qualityadjustment process appropriate for a portrait image is executedexclusively on a partial area that includes the subject area.
 16. Animage quality adjustment method according to claim 13 or 14, wherein thestep of executing an image quality adjusting process includes the stepof executing an image quality adjustment process appropriate for alandscape image when the image quality adjustment process appropriatefor a portrait image is not executed, and if a proportion of pixelshaving hue in a green range is greater than a second predeterminedthreshold value or if a proportion of pixels having hue in a sky bluerange is greater than a third predetermined threshold value.
 17. Animage quality adjustment method according to claim 13, wherein the imagequality adjustment is executed exclusively on a target processing areawhich is a portion of the image and which contains pixels present withinthe subject area and having color of the predetermined color range. 18.An image quality adjustment method according to claim 17, wherein thetarget processing area includes pixels of a first type present withinthe subject area and having color of the predetermined color range, andpixels of a second type present outside the subject area, but contiguouswith the pixels of the first type and having color of the predeterminedcolor range.
 19. A computer program for causing a computer to executeprocessing of image data, using image data generated by an imagegenerating device, and image generation record information that isassociated with the image data and that includes at least subject areainformation indicating a subject area in the image, the computer programcausing a computer to calculates a hue of each pixel in the subjectarea; and if a proportion of pixels having hue of a predetermined colorrange is greater than a first predetermined threshold value, thecomputer program causing the computer to execute an image qualityadjustment process appropriate for an image containing a subjectidentified by the predetermined color range.
 20. A computer-readablerecording medium having recorded thereon the computer program accordingto claim 19.